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[57] ABSTRACT 

A face image processing system capable of setting an eye 
detecting region with ease and high accuracy by making use 
of the position and the width of the nares presenting a unique 
shape among the face characterizing features. The system 
mentioned above includes a face image input means for 
inputting a face image, a binarizing means for bin arizing the 
face image as inputted, a characteristic element extracting 
means for extracting face characterizing elements from a 
binary image outputted from the binarizing means, a naris 
extracting means for extracting nares from the characteristic 
elements extracted, and an eye extracting means for setting 
an eye detecting region with reference to a reference posi- 
tion provided by the nares for thereby extracting the eyes. 
The naris extracting means is comprised of a naris-width 
threshold value setting means for setting a naris-width 
threshold value for extracting naris candidates, a naris 
candidate extracting means for extracting naris candidates 
by using the naris-width threshold value, a naris region 
specifying means for specifying a naris region on the basis 
of the naris candidates, and a reference position setting 
means for setting a reference position for the specified naris 
region. 

6 Claims, 16 Drawing Sheets 
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BACKGROUND OF THE INVENTION 

1. Field of the Invention 

The present invention generally relates to a face image 
processing system including a means for setting an eye 
detecting region on the basis of a naris reference position by 
resorting to an image processing. More specifically, ihe 
invention is concerned with a method of detecting a naris or 
nares and a naris reference position. 

2. Description of Related Art 

For having better understanding of the invention, back- 
ground techniques thereof will first be described in some 
detail. As the eye detecting region setting means known 
heretofore, there may be mentioned a system disclosed, for 
example, in Japanese Unexamined Patent Application Pub- 
lication No. 227278/1994 (JP-A-6-227278). FIG. 18 is a 
block diagram showing a generally a configuration of a 
whole system including an eyeball existing region setting 
means 47. 

Referring to FIG. 18, setting of an eyeball existing region 
by an eyeball existing region setting means 47 is effected 
within an face image 49 illustrated in FIG. 19 and obtained 
through a binarization processing executed by a binarizing 
means 48. More specifically, the eyeball existing region 
setting means 47 sets at first a search starting line 50 at a 
frame center within the binarized image, as shown in FIG. 
19. Starting from the search starting line 50, the number of 
white pixels existing continuously is counted leftwards and 
rightwards, respectively, whereon a leftmost x-coordinate 
(XLM) 51 and a rightmost x-coordinate (XRM) 52 are 
stored which correspond to the maximum counts in the left 
and right directions, respectively. On the basis of the left- 
most and rightmost coordinates, transverse widths of the 
eyeball existing regions are established in accordance with 
the following expressions: 

center coordinate of abscissa-XC-XLM+((?CRM-XLM)f2) 
leftmost x-coordinate S3L of a lerVeye window^ 
rightmost x-coordinate S3R of a left-eye windowR#2«%TC-25 
leftmost x-coordinate SAL of a right-eye window«vCVl«vSTC+25 
rightmost x-coordinate SAR of a right- eye window*%CK2.*XRAf 

Subsequently, longitudinal or vertical widths of the eye- 
ball existing regions are established in such a manner as 
described below. 

For the left eye as viewed in the figure, the starting point 
is set at a position (X2-10) shifted by 10 dots from the right 
x-coordinate X2 of the left-eye window and a search range 
is set between the points (X2-10) and (X2-90) in order to 
evade the detection of black areas corresponding to naris, as 
can be seen from FIGS. 20A and 20B. A search starting line 
55 (YL) is set between the scanning line having the left and 
right ends determined, whereon the scanning is performed in 
the horizontal or transverse direction periodically at an 
interval of four dots from the line YL to 0 in the vertical 
direction to thereby determine the bottom points (maximum 
y-coordinate points: BY1MAX and BY2MAX) of the first 
and second black regions as counted from the scan starting 
point and decides presence or absence of the spectacles. The 
y-coordinates determining the longitudinal or vertical width 
of the eyeball existing region can be given by the following 
expressions: 



top y-coordinate of the eye window- YT-BY1MAX-40 
bottom y-coordinatc of the eye window»yB=BYlMAX+10 

5 

<with spectacles> 

top y-coordinate of the eye window- JT=B Y2M AX-40 

jo bottom y-coordinate of the eye window=yB=BY2MAX+10 

Similarly, for the right-hand eye, the bottom points of the 
first and second black regions from the scan starting point 
are determined by scanning periodically at an interval of 
four dots in the transverse or horizontal direction and the 
presence or absence of the spectacles is determined, 
whereon the top and bottom coordinates of the right eye is 
established through the similar procedure as in the case of 
the left eye. FIG, 21 is a view showing an eye window 56 

2o set up through the procedure described above. 

In conjunction with the conventional face image process- 
ing system described above, it is noted that the binarized 
region of the image tends to change or vary in dependence 
on weather, orientation of the face and other factors. Thus, 
the binarized region is unstable. However, for the eyeball 

25 existing region setting means 47 described above, it is 
required that the face contour position and/or the black 
regions such as those corresponding to the eyes, eyebrows 
and others have to be detected with high accuracy for setting 

3Q the frame widths. Such detection may become difficult in 
dependence on the binarized states, possibly leading to 
setting of an erroneous eyeball existing region, giving rise to 
a problem. 

SUMMARY OF THE INVENTION 

35 

In the light of the state of the art described above, it is an 
object of the present invention to provide a face image 
processing system which is capable of setting an eye detect- 
ing region with enhanced accuracy by searching at high 
40 speed with high accuracy a naris or nares presenting a 
unique shape among the face characterizing elements or 
features and thus capable of setting a reference position for 
a naris region by making use of the positions and the widths 
of the nares. 

45 In view of the above and other objects which will become 
apparent as the description proceeds, there is provided 
according to a general aspect of the present invention a face 
image processing system which includes a face image input 
means for inputting a face image, a binarizing means for 

50 binarizing the face image as inputted, a characteristic 
element/feature extracting means for extracting face char- 
acterizing elements or features from a binary image output- 
ted from the binarizing means, a naris extracting means for 
extracting a naris from the characteristic elements as 

55 extracted, and an eye extracting means for setting an eye 
detecting region with reference to a reference position 
provided by the naris as extracted for thereby extracting the 
eye. The naris extracting means is constituted by a naris- 
width threshold value setting means for setting a naris-width 

60 threshold value for extracting of naris candidates, a naris 
candidate extracting means for extracting a naris candidate 
by using the naris-width threshold value, a naris region 
specifying means for specifying a naris region on the basis 
of the naris candidate, and a reference position setting means 

65 for setting a reference position for the specified naris region. 
In a preferred mode for carrying out the invention, the 
characteristic element/feature extracting means mentioned 
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above may include a y-histogram generating means for 
counting the number of black pixels in a face- transverse 
direction for the individual x-coordinates, respectively, 
within a predetermined region to thereby generate a 
y-histogram in which the black pixel number is taken along 
the ordinate with the x-coordinates being taken along the 
abscissa, a characteristic region setting means for searching 
a peak for which a maximal value of the black pixel numbers 
in the y-histogram is not smaller than a first predetermined 
value and setting for the peak a starting position and a 
terminal position at locations where the black pixel number 
is not smaller than a second predetermined value and where 
distance from the point of the maximal value is not smaller 
than a second predetermined value and not greater than a 
third predetermined value, and an x-histogram generating 
means for counting the number of the black pixels in a 
face-longitudinal direction for the individual y-coordinates, 
respectively, in the characteristic region extending from the 
starting position to the terminal position set by the charac- 
teristic region setting means to thereby generate an 
x-histogram in which the y-coordinates are taken along the 
ordinate with the black pixel number being taken along the 
abscissa, wherein the x-histogram generating means is fur- 
ther adapted for generating an x-profile histogram in which 
distances to the position of the black pixel making appear- 
ance finally in the aforementioned search from the terminal 
position to the starting position is taken along the abscissa 
with the y-coordinates being taken along the ordinate within 
the characteristic region. 

In another preferred mode for carrying out the invention, 
the naris-width threshold value setting means may set a 
maximum value of the x-histogram as a naris-width thresh- 
old value within each characteristic regions. 

In yet another preferred mode for carrying out the 
invention, the naris candidate extracting means may include 
a characteristic region binarizing means for binarizing the 
characteristic region with the aid of the x-histogram and the 
x-profile histogram generated by the x-histogram generating 
means, and a naris candidate searching means for searching 
binary data of the characteristic region by using the naris- 
width threshold value set by the naris-width threshold value 
setting means, to thereby establish a starting point and a 
terminal point for the naris candidate. 

In a further preferred mode for carrying out the invention, 
the naris region specifying means may include a first select- 
ing means for selecting a naris candidate from those 
extracted by the naris candidate extracting means on the 
basis of widths and positions of the naris candidates in the 
y-direction, and a second selecting means for selecting a 
naris candidate from those selected by the first selecting 
means on the basis of positions in the x-direction and areas 
of the naris candidates. 

In a yet further preferred mode for carrying out the 
invention, the reference position setting means may be so 
implemented as to arithmetically determine a center point 
between the naris to thereby set the center point as a 
reference position. 

The above and other objects, features and attendant 
advantages of the present invention will more easily be 
understood by reading the following description of the 
preferred embodiments thereof taken, only by way of 
example, in conjunction with the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

In the course of the description which follows, reference 
is made to the drawings, in which: 
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FIG. 1A is a schematic block diagrams showing a general 
arrangement of a face image processing system according to 
an embodiment of the present invention; 

FIG. IB is a functional block diagram showing major 
5 functional or processing means employed in the system 
according to the invention; 

FIG. 2 is a flow chart for illustrating generally a process- 
ing executed by a processor of the face image processing 
system shown in FIG. 1A; 
10 FIG. 3 is a pictorial view of a face image taken by an 
image pickup device in the system shown in FIG. 1A; 

FIG. 4 is a schematic view showing a binary image 
resulting from binarization processing of the face image 
15 shown in FIG. 3; 

FIGS. 5A, 5B, 5a, 5b, 5c and 5d are diagrams for 
illustrating operations of MAX/MIN filters which are incor- 
porated in a binarizing means of the system according to the 
invention; 

20 FIGS. 6A and 6B are schematic signal waveform dia- 
grams for illustrating operations of the binarizing means; 

FIG. 7 is a flow chart for illustrating generally processing 
steps carried out by a characteristic element/feature extract- 
ing means shown in FIG. IB; 

25 FIGS. 8A and SB are diagrams for graphically illustrating 
operation of a y-histogram generating means incorporated in 
a characteristic element/feature extracting means shown in 
FIG. IB; 

FIGS. 9A and 9B are diagrams for graphically illustrating 
30 operation of an x-histogram generating means incorporated 
in the characteristic element/feature extracting means; 

FIG. 10 is a view for illustrating operation of a naris- 
width threshold value setting means shown in FIG. IB; 
35 FIG. 11 is a flow chart illustrating processing steps 
executed by a naris candidate extracting means; 

FIGS. 12A, 12B and 12C are views illustrating operation 
of a characteristic region binarizing means according to an 
embodiment of the present invention; 
40 FIG. 13 is a view illustrating operation of a naris candi- 
date searching means according to an embodiment of the 
present invention; 

FIG. 14 is a flow chart for illustrating schematically 
processings executed by a naris region specifying means 
45 according to an embodiment of the invention; 

FIG. 15 is a view for illustrating operation of a first 
selecting means incorporated in the naris region specifying 
means; 

FIG. 16 is a view for illustrating operation of a second 
selecting means incorporated in the naris region specifying 
means; 

FIG. 17 is a view for illustrating operation of a reference 
position setting means incorporated in the naris region 
55 specifying means; 

FIG. 18 is a block diagram showing a generally a con- 
figuration of a conventional face image processing system 
known heretofore; 

FIG. 19 is a view for illustrating operation of the Con- 
go ventional face image processing system for setting a trans- 
verse or horizontal width of a eyeball existing region; 

FIGS. 20A and 20B are views for illustrating operation of 
the conventional face image processing system for setting a 
longitudinal or vertical width of a eyeball existing region; 
65 and 

FIG. 21 is a view showing eye windows as set in the 
conventional face image processing system. 
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DESCRIPTION OF THE PREFERRED (i.e., direction longitudinally of the face) within the charac- 

EMBODIMENTS terislic region extending from the starting position to the 

vt # , . . t . -11 u j u j • j * i ■ terminal position set by the characteristic region setting 

Now the present invention wi 11 be described in detail in means ^ tfaereb ^ M x . nist £ which ^ 

conjunction with what is presently considered as preferred y_c 00rdinates are taken al the oriin * ^ the black 

or typical embodiments thereof by reference to the drawings. pixel number ^ taken along the abscissa> wherein the 

In the following description, like reference characters des- x -histogram generating means 26 is further adapted for 

ignate like or corresponding parts throughout the several generating an x-profile histogram in which distances to the 

views. Also in the following description, it is to be under- position of the black pixel making appearance finally in 

stood that such terms as "left", "right", "top", "bottom", searching from the aforementioned terminal position to the 

"leftwards", "rightwards", "vertical", "horizontal" and the starting position are taken along the abscissa with the 

like are words of convenience and are not to be construed as y-coordinates being taken along the ordinate. 

limiting terms. The naris extracting means 7 is composed of a naris-width 

Embodiment 1 threshold value setting means 9 for setting a naris-width 

15 threshold value allowing naris candidates to be extracted, a 

FIG. 1A is a schematic block diagram showing a general naris candidate extracting means 10 for extracting naris 

arrangement of a face image processing system according to candidates by using the naris-width threshold value, a naris 

a first embodiment of the present invention. As can be seen region specifying means 11 for specifying a naris region on 

in FIG. 1A, the face image processing system according to the basis of the naris candidates as extracted, and a reference 

the instant embodiment of the invention is comprised of an 2Q position setting means 12 for setting a reference position for 

image pick-up device 1 such as a CCD (charge coupled the specified naris region. 

device) camera or the like for picking up face pictures or Th e naris candidate extracting means 10 includes a char- 
images, an image memory 2 for storing image data of the acteristic region binarizing means 29 for binarizing the 
face as outputted from the image pickup device 1 and a characteristic region with the aid of the x-histogram and the 
processor 3 such as a microprocessor or the like for per- 25 x -profile histogram generated by the x-histogram generating 
forming image processings on the basis of the data stored in means 26 and a naris candidate searching means 30 for 
the image memory 2. searching the binary data of the characteristic region by 
FIG. IB is a functional block diagram showing schemati- using the naris-width threshold value set by the naris-width 
cally and generally a system configuration of the face image threshold value setting means 9 to thereby set up a starting 
processing system according to the first embodiment of the 30 point and a terminal point for the naris candidate, 
invention. As can ben seen in FIG. IB, the face image The naris region specifying means 11 includes first select- 
processing system according to the instant embodiment of mg means 33 for selecting naris candidate from those 
the invention includes as functional units a face image input extracted by the naris candidate extracting means 10 on the 
means 4 for inputting a face image, a binarizing means 5 for basis of the width and the position in the y-direction, and a 
binarizing (i.e., digitizing into binary bits) the face image as 35 second selecting means 34 for selecting a naris candidate 
inputted, a characteristic element/feature extracting means 6 from those selected by the first selecting means 33 on the 
for extracting face characterizing elements or features from basis of the position in the x-direction and the area of the 
an binary image generated by the binarizing means 5, a naris naris candidate. 

extracting means 7 for extracting nares from the character- In operation, the number of the black pixels in the 

istic elements or features extracted by the characteristic 40 f aC e-transverse direction relative to the face is counted by 

element/feature extracting means 6, and an eye detecting the y-histogram generatmg means 23 vvithin a predetermined 

region setting means 8 for setting an eye detecting region region for the md i vidua i x-coordinates, respectively, to 

with reference to the extracted nares for allowing the eye thereby gene rate the y-histogram in which the black pixel 

image to be extracted. number is taken along the ordinate with the x-coordinates 

Generally, the characteristic element/feature extracting 45 being taken along the abscissa, as is illustrated in FIG. SA. 

means 6 is comprised of a y-histogram generating means 23, Subsequently, the peaks for which a maximal value point of 

a characteristic region setting means 25 and an x-histogram the black pixel numbers is not smaller than the first prede- 

generating means 26, which will be described below in more termined value 24a are searched in the y-histogram to 

detail, thereby set for each of the peaks the starting position 24c/ 

More specifically, the characteristic element/feature 50 and the terminal position 24di at locations where the black 

extracting means 6 includes a y-histogram generating means pixel number exceeds a second predetermined value 24e 

23 for counting the number of black pixels at the individual inclusive and where the distance from the maximal point 

x-coordinates along a face-transverse direction (i.e., trans- value is not greater than the third predetermined value 24b 

verse direction as viewed toward the face) within a prede- by the characteristic region setting means 25, as is illustrated 

termined region to thereby generate a y-histogram in which 55 in FIG. 8B. In succession, the black pixel number within the 

the black pixel number is taken along the ordinate with the characteristic region 27 ai delimited by the starting position 

x-coordinates being taken along the abscissa, a characteristic 24c/ and the terminal position lAdi as set up are counted by 

region setting means 25 for searching a peak for which a the x-histogram generating means 26 for each of the 

maximal value of the black pixel numbers is not smaller than y-coordinates in the face-longitudinal direction to thereby 

a first predetermined value in the y-histogram to thereby set 60 generate the x-histogram in which the y-coordinates are 

for the peak a starting position and a terminal position at taken along the ordinate with the black pixel number being 

locations where the black pixel number is not smaller than taken along the abscissa, as can be seen in FIG. 9 A. 

a second predetermined value and distance from the maxi- Moreover, the x-profile histogram is generated in which the 

mal value point is not greater than a third predetermined distances m to the position of the black pixel making 

value, and an x-histogram generating means 26 for counting 65 appearance finally in the search performed from the terminal 

the number of the black pixels for the individual position 24di toward the starting position 24d are taken 

y-coordinates, respectively, in a face-longitudinal direction along the abscissa with the y-coordinates being taken along 



04/30/2003, EAST version: 1.03.0002 



6,055,323 



8 



10 



15 



20 



25 



30 



the ordinate within the characteristic region 21ai, as is 
illustrated in FIG. 9B. 

FIG. 2 is a flow chart for illustrating generally processings 
executed by the processor 3 of the face image processing 
system shown in FIG. 1A. Referring to FIG. 2, the image 
data of the face image are inputted to the processor 3 from 
the image memory 2 through the medium of the face image 
input means 4 (step ST1), whereon the face image as 
inputted is binarized (i.e., digitized into binary bits) by the 
binarizing means 5. The binary image data thus obtained are 
then stored in the image memory 2 (step ST2). 
Subsequently, the face characterizing elements or features 
are extracted from the binary image data by the character- 
istic element/feature extracting means 6 (step ST3), whereon 
the naris-width threshold value is established for allowing 
the naris candidates to be extracted by the naris-width 
threshold value setting means 9 (step ST4). In succession, 
the naris candidates are extracted by the naris candidate 
extracting means 10 by referencing the naris-width threshold 
value as established (step ST5), to thereby specify a naris 
region from the extracted naris candidates by the naris 
region specifying means 11 (step ST6). Next, the reference 
position for the naris region is set up by the reference 
position setting means 12 (step ST7) while the eye detecting 
region is established by the eye detecting region setting 
means 8 (step ST8). Details of the individual processing 
steps will be described below. 

FIG. 3 is a pictorial view of a face image 12A taken by the 
image pick-up device 1. Further, FIG. 4 is a schematic view 
showing a binary image 13 resulting from the binarization of 
the face image 12A. Furthermore, FIGS. 5A, 5B, 5a, 5b, 5c 
and 5d are diagrams for illustrating operations of a MAX 
filter and a MIN filter (hereinafter also referred to collec- 
tively MAX/MIN filter) which are incorporated in the bina- 
rizing means 5. Additionally, FIGS. 6A and 6B are signal 
waveform diagrams for illustrating the operation of the 
binarizing means 5. 

Before entering into description of the binarizing means 
5, operations of the MAX/MIN filter will be elucidated. 
Shown in FIG. 5 A is a waveform of a portion of an input 
picture signal (original image signal) 14, while shown in 
FIG. SB is an image signal 15 obtained after processing the 
original image signal 14 by the MAX filter. Further shown 
in FIG. 5a is an image signal 16 resulting from the MIN- 
filter processing of the image signal 15 outputted from the 
MAX filter processing. In FIG. 5b, reference numerals 17 to 
19 designate pixel-based array of luminance values of the 
original image signals 14, 15 and 16, respectively, which 
correspond, respectively, to the individual pixels shown 
schematically. For the convenience of illustration, it is so 
presumed that a given one frame enclosing a luminance 
value represents one pixel and that the luminance values lie 
within a range of "0" to "21". 

At first, the original image signal 14 undergoes the MAX 
filter processing. Parenthetically, it should be mentioned that 
the MAX filter has a predetermined length (a predetermined 
number of pixels a which will also be referred to as the filter 
size) and serves for converting the luminance value of the 
pixel located at the center of the filter into a maximal 
luminance value in the filter. By way of example only, let's 
assume that the pixel number a is "5" and that hatched pixels 
designated by reference numeral 17 shown in FIG. 5b 
undergo the MAX filter processing. In that case, the maxi- 
mal luminance value of the pixels shown by hatching is "9". 
Accordingly, through the MAX filter processing, the lumi- 
nance value of a pixel 11a located at the center of the MAX 
filter is translated from "T to "9". Subsequently, the hatched 
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pixel region is shifted to the right by one pixel and under- 
goes the MAX filter processing. By repeating this process, 
the array of the luminance values designated by the refer- 
ence numeral 17 in FIG. 5b is transformed into the lumi- 
nance value array designated by the numeral 18 in FIG. 5b. 
In succession to the MAX filter processing, the image signal 
outputted therefrom is subjected to the MIN filter process- 
ing. 

The MIN filter has a same filter size as that of the MAX 
filter and serves to convert the luminance value of the center 
pixel of the filter into the minimal luminance value in the 
filter. The hatched pixel region of the luminance value array 
obtained from the MAX filter processing as designated by 
the reference number 18 undergoes the MIN filter process- 
ing. In that case, the minimal luminance value of the pixels 
shown by hatching is "9". Accordingly, through the MIN- 
filter processing, the luminance value of pixel ISb located at 
the center of the MAX filter is translated from "10" to "9". 
Subsequently, the hatched pixel region is shifted to the right 
by one pixel and undergoes the MIN-filter processing. By 
repeating this process, the array of the luminance values 
designated by the reference numeral 18 in FIG. 5b is 
transformed into the luminance value array designated by 
the numeral 19 in FIG. 5b. By subtracting the input image 
from the image resulting from the MIN/MAX filter 
processing, there can be extracted a low luminance region of 
a width not greater than the filter size. In the case of the 
instant example, the luminance value array 17 is subtracted 
from the array 19 shown in FIG. 5b, there is obtained a pixel 
array 19A shown in FIG. Sd as the difference resulting from 
the subtraction, the difference being shown in FIG. 5c in the 
form of image signal. As can be seen from FIG. 5, in both 
the MAX filter processing and the MIN filter processing, the 
conversion or translation begins from the center pixel of the 
array subjected to the filtering at first and ends at the center 
pixel of the array undergone the filtering finally. 
Consequently, the data obtained after the MAX filter pro- 
cessing is composed of the pixel array having the leading 
and trailing pixels denoted by 18c and lSd, respectively, as 
shown in FIG. 5b, while the data resulting from the MIN 
filter processing is composed of the pixel or luminance value 
array starting at 19e and terminating at 19/ as shown in FIG. 
5b. Thus, the image resulting from the MAX/MIN filter 
processings is reduced by (a-1) pixels at the processing 
starting and terminating positions, respectively. 

In the face image processing system according to the 
instant embodiment of the invention, the image processing 
is performed on a row-by-row basis in the longitudinal 
direction as indicated by an arrow 20 in FIG. 4. Referring to 
FIG. 6 A, there is illustrated an image signal 21 obtained 
after the MAX/MIN filter processings of lengths corre- 
sponding to the widths of eyebrow and eye images contained 
in the input picture signal. In the case of the example shown 
in FIG. 6 A, images of a nose, a mouth and others having 
widths narrower than those of the eyebrow and the eye are 
extracted in addition to the latter. As can be seen in FIG. 6A, 
by contrast, the face featuring points such as the eyebrow, 
the eye, the nose and the mouth have high luminance values, 
respectively, after the differentiation processing, as indicated 
by waveforms 21a. By contrast, images of shadow and cloth 
are low in the luminance as indicated by a waveform 216. 
Such being the circumstances, a threshold value such as 
indicated by a reference number 22 is established, where- 
upon the binarization processing of the image data of the 
features is performed. An image obtained through the pro- 
cessings described above is illustrated in FIG. 4. In con- 
junction with the image shown in FIG. 4, solid regions will 
be referred to as the black regions. 
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Now, the description will be directed to the characteristic 
element/feature extracting means 6. FIG. 7 is a flow chart for 
illustrating generally the processing carried out by the 
characteristic element/feature extracting means 6, and FIGS. 
8 A, 8B, 9A and 9B are diagrams for graphically illustrating 
operation or function of the characteristic element/feature 
extracting means 6. 

Now referring to FIG. 7, description will be directed to the 
processings executed by the characteristic element/feature 
extracting means 6. At first, the number of the black pixels 
in the face-transverse direction is counted by the 
y-histogram generating means 23 in the predetermined 
region for the individual x-coordinates, respectively, to 
thereby generate the y-histogram in which the black pixel 
number is taken along the ordinate with the x-coordinates 
being taken along the abscissa (step ST23), as is illustrated 
in FIG. 8 A. Subsequently, the peak values for which a 
maximal value point of the black pixel numbers is not 
smaller than the first predetermined value 24a are searched 
in the y-histogram to thereby set for each of the peaks the 
starting position 24c/ and the terminal position 24di at 
locations where the black pixel number is not smaller than 
the second predetermined value 24e inclusive and the dis- 
tance from the maximal point value is not greater than the 
third predetermined value 24b by the characteristic region 
setting means 25 (step ST25), as is illustrated in FIG. SB. In 
succession, the black pixel number within the characteristic 
region 27ai delimited by the starting position 24c/ and the 
terminal position 24di as set up are counted by the 
x-histogram generating means 26 for each of the 
y-coordinates in the face-longitudinal direction to thereby 
generate the x-histogram in which the y-coordinates are 
taken along the ordinate with the black pixel number being 
taken along the abscissa, as can be seen in FIG. 9A. 
Moreover, the x-profile histogram is generated in which the 
distances m to the positions of the black pixel making 
appearance finally in the searches performed from the ter- 
minal position 24di toward the starting position 24c/ are 
taken along the abscissa with the y-coordinates being taken 
along the ordinate (step ST26), as is illustrated in FIG. 9B. 

Next, description will turn to the naris-width threshold 
value setting means 9 and the naris candidate extracting 
means 10. Referring to FIG. 10, the naris-width setting 
means 9 searches the maximum value of the x-histogram 
within each of the characteristic regions to thereby set the 
maximum value as a naris-width threshold value (S-WD) 
28. 

Now, the naris candidate extracting means 10 will be 
described. The naris candidate extracting means 10 includes 
the characteristic region binarizing means 29 and the naris 
candidate searching means 30, as mentioned hereinbefore by 
reference to FIG. IB. FIG. 11 is a flow chart for illustrating 
the processing executed by the naris candidate extracting 
means 10. Referring to FIG. 11, the characteristic region 
binarizing means 29 is so designed as to binarize each of the 
characteristic regions (step ST29) and then search the naris 
candidates by the naris candidate searching means 30 (step 
ST30). More specifically, the characteristic region binariz- 
ing means 29 is so implemented as to arithmetically deter- 
mine an overlap 31 of the black regions in the x-direction 
between the currently scanned line and the previously 
scanned line on the basis of the x-histogram and the x-profile 
histogram both generated by the x-histogram generating 
means 26 in accordance with the expressions mentioned 
below in which the value of the x-histogram for the 
y -coordinate value of a is represented by Ha with the value 
of the x-profile histogram being represented by HKa. 
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When HKa>HKa-l > overlap ZU(HKa-\-Xa) (refer to FIG. 12A) 
When HKa<HKa-l, overlap n~(HKa-Xa-l) (refer to FIG. 12B) 

In the above expressions, Xa=HKa-Ha, Xa-l=HKa-l- 
Ha-1. 

The region is binarized by executing noise correction in 
the y-direction with "1" when the overlap 31 determined 
from the above expression is positive polarity (see FIGS. 
12A and 12B) and, if otherwise, "0" is set (see FIG. 12C). 

In the naris candidate searching means 30, the naris-width 
threshold value (S_WD) 28 set by the naris-width threshold 
value setting means 9 is used for determining the starting 
point 32s/ and the terminal point 32W of the region such that 
a continuous number 35, i.e., the number of binary data "1" 
outputted from the characteristic region binarizing means 29 
continuously in the y-direction, satisfy the following con- 
ditions: 



2Q continuous number 35>(£_WD/2) 

continuous number 3S<(S_WD/2) 

The starting point 32s/ and the terminal point 32ei deter- 
mined in this manner are shown in FIG. 13. 

25 Next, description will be made of the naris region speci- 
fying means U. In the case where more than two naris 
candidates exist in the characteristic region, the naris region 
specifying means 11 sets naris regions for these naris 
candidates. The naris region specifying means 11 includes 

30 the first selecting means 33 and the second selecting means 
34, as shown in FIG. IB. FIG. 14 is a flow chart for 
illustrating generally the processings executed by the first 
selecting means 33 and the second selecting means 34. 
When an inter-naris width 38 between the naris candidates 

35 36 and 37 paired in the y-direction (see FIG. 15) satisfies the 
following conditions 

inter-naris width 38<(S_WD*2) 
inter-naris width 38>(S_WD/2) 

40 

and when absolute value 40=|39C1-39C2|, i.e., the absolute 
value of the difference between the center point 39C2 
located at the center between the top end of the naris 
candidate 36 and the bottom end of the naris candidate 37 as 
45 viewed in the y-direction and the center point 39C1 located 
at the center between the bottom end of the naris candidate 
36 and the top end of the naris candidate 37 as viewed in the 
y-direction assumes a minimum value, the first selecting 
means 33 selects the naris region candidate 41, whereon the 
processing is transferred to the second selecting means 34. 
When the x-profile histogram values 42S; 42E and 43S; 43E 
for the starting points 32si and the terminal points 32e/ of the 
naris candidates 36 and 37 within the naris region candidate 
41 and the maximum values 44M1 and 44M2 of the x-profile 
histograms of the naris candidates 36 and 37 (see FIG. 16) 
satisfy the following conditions 
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60 42E<44M1 
43S<44M2 
43E<44M2 
|44Ml-44M2j<10 

and when the absolute value 46«|45<wil-45ad2|, i.e., the 
absolute value of the difference between areas 45ad\ and 
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45ad2 of the naris candidates 36 and 37, respectively, 
assumes the minimum value (see FIG. 16), the second 
selecting means 34 selects the naris region candidate 41 as 
the naris region, which is then followed by the setting of the 
reference position by the reference position setting means 5 
12, whereupon the processing is transferred back to the first 
selecting means 33. Subsequently, the procedure described 
above is continued for all the characteristic regions and upon 
completion of the processing, the reference position is 
updated. To this end, the reference position setting means 12 
sets the point (Xc, Yc) as the reference point 46, where Xc 
represents the center point between the starting position 24ci 
and the terminal position 24di of the characteristic region in 
which the naris region exists and Yc represents the center 
point 39C1 mentioned above (see FIG. 17). 

The eye detecting region setting means 8 in turn sets the 15 
eye detecting region on the basis of the reference point 46 as 
in the case of the conventional system described hereinbe- 
fore by reference to FIGS. 18 to 21. 

As will now be apparent from the foregoing description, 
with the arrangement of the face image processing system 20 
according to the invention, the naris having a unique shape 
among the face characterizing elements or features can be 
detected at high speed with enhanced accuracy. Besides, 
because the reference position for the naris region can be set 
with high accuracy by making use of the position and the 25 
width of the naris, the eye detecting region can be set with 
high accuracy on the basis of the naris region. 

Additionally, because the processing is performed after 
the extraction of the face characterizing elements or features 
in succession to the binarization of the face image taken by 
the pick-up device, the image processing can be simplified 30 
and thus carried out at a high speed. 

Furthermore, by performing again the binarization pro- 
cessing after the extraction of the face characterizing ele- 
ments or features, overlaps between the naris regions as well 
as noise can be satisfactorily eliminated, whereby the naris 35 
can be detected with further enhanced accuracy. 

Many modifications and variations of the present inven- 
tion are possible in the light of the above techniques. It is 
therefore to be understood that within the scope of the 
appended claims, the invention may be practiced otherwise 40 
than as specifically described. 

What is claimed is: 

1. A face image processing system, comprising: 
a face image inputter for inputting a face image; 
a binary converter for binarizing the face image as input- 45 
ted; 

a characteristic element extractor for extracting face char- 
acterizing elements from a binary image outputted from 
said binary converter; 
a naris extractor for extracting nares from said face 50 
characterizing elements extracted from said character- 
istic element extractor, including: 
a naris-width threshold value discriminator for setting 
a naris-width threshold value for extracting naris 
candidates, said discriminator being operative to 55 
search for the maximum value of an x-histogram 
within a plurality of characteristic regions in deter- 
mining said naris-width threshold value; 
a naris candidate extractor for extracting naris candi- 
dates by using said naris-width threshold value; 60 
a naris region discriminator for specifying a naris 

region on the basis of said naris candidates; 
a reference positioner for setting a reference position 

for said specified naris region; and 
an eye extractor for setting an eye detecting region with 65 
reference to said reference position provided by said 
nares for extracting the eyes. 
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2. A face image processing system, comprising: 
a face image inputter for inputting a face image; 

a binary converter for binarizing the face image as input- 
ted; 

a characteristic element extractor for extracting face char- 
acterizing elements from a binary image outputted from 
said binary converter; 
a naris extractor for extracting nares from said face 
characterizing elements extracted from said character- 
istic element extractor, including: 
a naris-width threshold value discriminator for setting 
a naris-width threshold value for extracting naris 
candidates; 

a naris candidate extractor for extracting naris candi- 
dates by using said naris-width threshold value; 
a naris region discriminator for specifying a naris 

region on the basis of said naris candidates; 
a reference positioner for setting a reference position 

for said specified naris region; and 
an eye extractor for setting an eye detecting region with 
reference to said reference position provided by said 
nares for extracting the eyes, 
wherein said characteristic element extractor includes: 
a y-histogram generator for counting the number of 
black pixels in a face-transverse direction for 
individual x-coordinates, respectively, within a 
predetermined region to thereby generate a 
y-histogram in which the black pixel number is 
taken along the ordinate with the x-coordinates 
being taken along the abscissa; 
a characteristic region discriminator for searching a 
peak for which a maximal value of the black pixel 
numbers in said y-histogram is not smaller than a 
first predetermined value to thereby set for each of 
the peaks a starting position and a terminal posi- 
tion at locations where the black pixel number is 
not smaller than a second predetermined value and 
where distance from the maximal value point is 
not smaller than a second predetermined value and 
not greater than a third predetermined value; and 
a x-histogram generator for counting the number of 
the black pixels in a face-longitudinal direction for 
the individual y-coordinates, respectively, within 
the characteristic region extending from said start- 
ing position to said terminal position set by said 
characteristic region discriminator to thereby gen- 
erate an x-histogram in which the y-coordinates 
are taken along the ordinate with the black pixel 
number being taken along the abscissa, 
said x-histogram generator being further adapted for 
generating an x-profile histogram in which dis- 
tances to the position of the black pixel making 
appearance finally in the search from said terminal 
position to said starting position within said char- 
acteristic region is taken along the abscissa with 
the y-coordinates being taken along the ordinate. 

3. A face image processing system according to claim 2, 
wherein said naris-width threshold value discriminator 

sets a maximum value of said x-histogram within said 
characteristic region as a naris-width threshold value. 

4. A face image processing system according to claim 2, 
wherein said naris candidate extractor includes: 

a characteristic region binary converter for binarizing said 
characteristic region with the aid of said x-histogram 
and said x-profile histogram generated by said 
x-histogram generator; and 
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a naris candidate searcher for searching binary data in said 
characteristic region by using the nariswidth threshold 
value set by said naris-width threshold value discrimi- 
nator to thereby set up a starting point and a terminal 
point for the naris candidate. 5 

5. A face image processing system comprising: 

a face image inputter for inputting a face image; 

a binary converter for binarizing the face image as input- 
ted; 

a characteristic element extractor for extracting face char- 
acterizing elements from a binary image outputted from 
said binary converter; 
a naris extractor for extracting nares from said face 
characterizing elements extracted from said character- ^ 
istic element extractor, including: 
a naris-width threshold value discriminator for setting 
a naris-width threshold value for extracting naris 
candidates; 

a naris candidate extractor for extracting naris candi- 2 o 
dates by using said naris-width threshold value; 

a naris region discriminator for specifying a naris 
region on the basis of said naris candidates; 



a reference positioner for setting a reference position 

for said specified naris region; and 
an eye extractor for setting an eye detecting region with 
reference to said reference position provided by said 
nares for extracting the eyes, 
wherein said naris region discriminator includes: 
a first naris selector for selecting a naris candidate 
from those extracted by said naris candidate 
extractor on the basis of widths and positions of 
the naris candidates in the y-direction; and 
a second naris selector for selecting a naris candidate 
from those selected by said first naris selector on 
the basis of positions in the x-direction and areas 
of said naris candidates. 
6. A face image processing system according to claim 1, 
wherein said reference position discriminator arithmetically 
determines a center point between said nares to set said 
center point as said reference position. 
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